System and program for error handling in a dual adaptor system where one adaptor is a master

ABSTRACT

Provided is a method, system, and program for processing Input/Output (I/O) requests to a storage network including at least one storage device and at least two adaptors, wherein each adaptor is capable of communicating I/O requests to the at least one storage device. An error is detected in a system including a first adaptor, wherein the first adaptor is capable of communicating on the network after the error is detected. In response to detecting the error, a master switch timer is started that is less than a system timeout period if the first adaptor is the master. An error recovery procedure in the system including the first adaptor would be initiated after the system timeout period has expired. An operation is initiated to designate another adaptor in the storage network as the master if the first adaptor is the master in response to detecting an expiration of the master switch timer.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of application Ser. No. 11/929,442filed on Oct. 30, 2007, which is incorporated herein by reference in itsentirety.

Application Ser. No. 11/929,442 is a continuation of application Ser.No. 11/158,802, filed on Jun. 22, 2005, which is incorporated herein byreference in its entirety.

Application Ser. No. 11/158,802, is a continuation of application Ser.No. 09/998,463 filed on Nov. 29, 2001, which is also incorporated hereinby reference in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a system, method, and program for errorhandling in a dual adaptor system.

2. Description of the Related Art

In a storage loop architecture, such as the Serial Storage Architecture(SSA), a plurality of disks are interconnected to one or more adaptorsso that either of the adaptors can access the one or more loops ofinterconnected disks. An adaptor may include two or more ports to allowconnection to one or more loops. For each loop on which the adaptorcommunicates, one adaptor port connects to a first disk in the loop andthe other port connects to another disk in the loop. Additional adaptorsmay be added to the loop, such that one port on each other adaptorconnects to one disk and another port connects to another disk so thatthe additional adaptors are placed within the loop. Additional detailsof the SSA architecture and different possible loop topologies aredescribed in the International Business Machines Corporation (IBM)publication “Understanding SSA Subsystems in Your Environment”, IBMdocument no. SG24-5750-00 (April, 2000), which publication isincorporated herein by reference in its entirety.

One or more computer systems, such as storage subsystems, host system,etc., may include the adaptors connecting to the loop. Adaptors thatshare a loop must intercommunicate to coordinate accesses to disks inthe shared loop. High end storage systems, such as the IBM EnterpriseStorage Server (ESS), can detect errors in the ability of an adaptor inanother system to communicate with the local operating system eventhough such detected adaptor is still capable of communicating on thenetwork. In such instances, the system detecting the problem will delayI/O processing for a timeout period that corresponds to the timerequired for the other system including the adaptor to initiate an errorrecovery procedure. This timeout period must take into account alldifferent timeout periods and error recovery procedures that could occurwithin the detected system unable to communicate with the adaptor. Inmany cases the timeout period can extend for several minutes.

In storage systems requiring high availability, such as storage systemsfor critical uses, any delays in I/O processing are generallyunacceptable. Thus, extensive delays in I/O processing, such as a delayresulting from the lengthy timeout period for the error recovery processat the detected system, would be unacceptable in a high availabilitysystem.

In addition to delays that may result from having to wait for the systemhousing the other adaptor to reset, additional delays may be incurredwhen a master adaptor is subject to the reset. The master adaptor, whichis the configurator with the highest unique identifier (ID), isresponsible for configuring each port in the network with variousparameters and coordinating the processing of asynchronous events suchas dynamic changes in the network configuration. If a master adaptor isreset, then in the SSA architecture, the adaptor having the next highestunique identifier will be designated as the master. Followingreassignment of the master node, each remaining adapter on the loopadjusts internal routing algorithms under direction from the new masterinitiator, so that frames are automatically rerouted to avoid the break.This allows devices to be removed or added to the loop while thesubsystem continues to operate without interruption.

Upon resetting an adaptor, the system will experience a brief I/O delayto coordinate the reset adaptor entering a disabled state. If a slave isreset, then the I/O delay may only be a few seconds. However, if themaster is reset, then the I/O delay may double to 8 to 16 seconds due tothe additional time needed to switch the master to another adaptor.

For these reasons there is a need in the art to provide improved errorhandling that reduces timeout delays in systems where two adaptors arecapable of accessing the storage devices and reduces delays associatedwith resetting the master adaptor.

SUMMARY OF THE PREFERRED EMBODIMENTS

Provided is a method, system, and program for processing Input/Output(I/O) requests to a storage network including at least one storagedevice and at least two adaptors, wherein each adaptor is capable ofcommunicating I/O requests to the at least one storage device. An erroris detected in a system including a first adaptor, wherein the firstadaptor is capable of communicating on the network after the error isdetected. In response to detecting the error, a master switch timer isstarted that is less than a system timeout period if the first adaptoris the master. An error recovery procedure in the system including thefirst adaptor would be initiated after the system timeout period hasexpired. An operation is initiated to designate another adaptor in thestorage network as the master if the first adaptor is the master inresponse to detecting an expiration of the master switch timer.

In further implementations, a reset request is sent to the first adaptorafter the master switch timer expires.

In still further implementations, in response to detecting the error, amonitoring state is initiated to monitor I/O requests transmittedthrough a second adaptor. In response to receiving an I/O request, anI/O delay timer is started that is less than the system timeout period.A reset request is sent to the first adaptor in response to detecting anexpiration of one started I/O delay timer.

Yet further, the steps of initiating a monitoring state, starting theI/O delay timer and sending the reset request may be performed by adevice driver executing in an operating system.

The described implementations provide improved error handling techniqueswhen detecting a problem with the ability of another adaptor tocommunicate to the system in which the adaptor is housed. The describederror handling techniques reduce delays in I/O processing when detectingerrors in the ability of other adaptors to communicate to the system inwhich they are housed.

BRIEF DESCRIPTION OF THE DRAWINGS

Referring now to the drawings in which like reference numbers representcorresponding parts throughout:

FIG. 1 illustrates a computing environment in which aspects of theinvention are implemented; and

FIG. 2 illustrates an alternative computing environment in which aspectsof the invention are also implemented; and

FIG. 3 illustrates logic implemented when detecting a problem in thesystem housing another adaptor in accordance with implementations of theinvention; and

FIGS. 4 a and 4 b illustrate additional logic implemented when detectinga problem in the system housing another adaptor in accordance withimplementations of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following description, reference is made to the accompanyingdrawings which form a part hereof and which illustrate severalembodiments of the present invention. It is understood that otherembodiments may be utilized and structural and operational changes maybe made without departing from the scope of the present invention.

FIG. 1 illustrates a computing environment in which aspects of theinvention are implemented. Computer systems 2 a and 2 b include at leastone adaptor 4 a and 4 b to enable communication with a loop 6 ofinterconnected storage devices 8 a, 8 b . . . 8 j, such as hard diskdrives, tape drives, optical disk drives, etc. Each adaptor 4 a and 4 bincludes a plurality of ports 10 a, 12 a, 14 a, 16 a and 10 b, 12 b, 14b, and 16 b, respectively. In FIG. 1, ports 10 a, 10 b, 12 a, and 12 bare shown as connected to the loop 6. The other ports 14 a, 14 b, 16 a,and 16 b may connect to additional loops. Further, FIG. 1 provides anexample loop configuration. Those skilled in the art will appreciatethat there are numerous loop configurations that may be implementedbetween one or more adaptors.

The systems 2 a and 2 b may comprise any computing device known in theart, such as a host system, storage subsystem, workstation, server, etc.Each system 2 a and 2 b includes an operating system 20 a and 20 b,respectively, and device drivers 22 a and 22 b that enable communicationbetween the operating systems 20 a, 20 b and the adaptors 4 a, 4 b. Eachof the storage devices 8 a, 8 b . . . 8 j would also include ports (notshown) to connect to the loop 6. In certain implementations, the loop 6may be implemented using the Storage Serial Architecture (SSA) or anyother protocol enabling multiple hosts to communicate with multiplestorage device, such as Fibre Channel, Ethernet, Gigabyte Ethernet,Infiniband, etc. The loop 6 may be part of a Storage Area Network (SAN),Wide Area Network (WAN), Local Area Network (LAN), the Internet, anIntranet, etc.

FIG. 2 illustrates an alternative implementation where the adaptors 54 aand 54 b are housed in different Input/Output bays 52 a and 52 b withinthe same system 50. Each I/O bay 52 a and 52 b provides a separate powerboundary. Thus, adaptors 54 a and 54 b are on separate power boundaries.Each adaptor 54 a and 54 b includes ports 60 a, 62 a, 64 a, 66 a and 60b, 62 b, 64 b, and 66 b, respectively, for communication with one ormore loops (not shown) according to any loop topology known in the art.In the implementation of FIG. 2, the system 50 includes one operatingsystem 70 and separate device drivers 72 a and 72 b to manage theoperations of adaptors 54 a and 54 b. Alternatively, only one instanceof the device driver may execute to manage the operations of both theadaptors 54 a and 54 b. Each I/O bay 52 a and 52 b provides differentcommunication channels and power sources for the adaptors 54 a and 54 b,respectively.

FIG. 3 illustrates logic implemented in the device drivers 22 a and 22 b(FIG. 1), as well as device drivers 72 a and 72 b (FIG. 2) to handle asituation where an error is detected in the system housing the othertarget adaptor 4 b. For instance, the target adaptor 4 b may be unableto communicate with the operating system 20 b due to a problem in thebus interfacing between the adaptor 4 b and the system 2 b in which theoperating system 20 b is executing. For instance, the adaptor 4 b may beimplemented on a Peripheral Component Interconnect (PCI) card and thePCI bus between the system 2 b and adaptor 4 b may be malfunctioning. Insuch cases, the target adaptor 4 b itself is operational, but does notfully function because of another problem in the system 2 b housing thetarget adaptor 4 b.

The error handling process of FIG. 3 begins at block 100 where thedevice driver in one adaptor, e.g., device driver 22 a in adaptor 4 a,detects an error within the system 2 b including the target adaptor 4 b,even though no error was detected at the target adaptor 4 b. In suchcase, the source adaptor 4 a would be able to continue processing I/Osagainst the storage devices 8 a, 8 b . . . 8 j because the targetadaptor 4 b is available on the loop 6. In response to detecting theerror, the device driver 22 a begins two concurrent (or non-concurrent)processes at blocks 102 and 106. At block 102, the device driver 22 astarts a window timeout timer 102. In certain implementations, theduration of the window timeout timer coincides with a time during whichthe system 2 b including the target adaptor 4 b is guaranteed to havecompleted an error recovery procedure. This window timeout timerincludes the time that would pass before the other system 2 b wouldinitiate the error recovery procedure after detecting a problem. Theerror recovery procedure initiated by the system 2 b would likelyeliminate the detected error. Upon the system 2 b beginning the errorrecovery procedure by resetting or undergoing a power cycle, the targetadaptor 4 b would go offline and not participate in the loop 6. Incertain implementations, each adaptor 4 a, 4 b on the loop is aware ofall active adaptors. Once an adaptor becomes unavailable or unable tocommunicate on the loop 6, then the adaptors remaining on the loop arenotified of such state change and remove the unavailable adaptor fromloop registries (not shown) each adaptor maintains that indicatesavailable adaptors on the loop. The capability of one system to detecterrors in other systems, such as problems a remote system may have incommunicating with internal adaptors is known in the art of highavailability storage subsystems, such as the IBM Enterprise StorageServers (ESS).** Thus, if the device driver 22 a, 22 b takes no actionfor the window timeout period, the device driver 22 a, 22 b can beassured that the other system 2 b will have reset and the communicationproblem with the target adaptor 4 b will have been repaired or takenoff-line. Thus, after an alert is received (at block 104) that thewindows timeout timer expired, the monitoring ends and any processes ortimers started in response to receiving an I/O request at block 104 areterminated (at block 105) because the communication problem in thesystem 2 b including the target adaptor 4 b is in the process of beingrepaired. During the system 2 b error recovery procedure, the targetadaptor 4 b would go off-line and not affect loop operations. **IBM andEnterprise Storage Server are trademarks of International BusinessMachines Corporation.

At block 106, upon receiving an Input/Output (I/O) request, the devicedriver 22 a starts (at block 108) an I/O delay timer for the receivedI/O request and forwards the I/O request to the adaptor 4 a to executeagainst the target storage device 8 a, 8 b . . . 8 j. In certainimplementations, the I/O delay timer reflects a maximum tolerable I/Odelay dependent on the environment in which the storage system isdeployed. Thus, the I/O delay timer in high availability systems isrelatively shorter than systems requiring less availability. Forinstance, in high availability systems, the windows timeout period mayrun for several minutes to ensure the initiation of the error recoveryprocess at the remote system 2 b and the I/O delay period may be only acouple of seconds. Upon receiving (at block 110) notification that themonitored I/O request completed, the device driver 22 a would stop (atblock 112) the I/O delay timer that was started for the completed I/Orequest and control returns to block 106 to await any further receivedI/O requests or one of the other events at blocks 104 and 114. Uponreceiving (at block 114) notification that one I/O delay timer expired,the device driver 22 a issues (at block 116) a command to the localadaptor 4 a to get the identifier (ID) of the target adaptor 4 b. Asmentioned, in certain implementations, the local adaptor 4 a couldaccess the target adaptor 4 b from a loop registry. If there is no IDfor the target adaptor 4 b in the loop registry, then that targetadaptor 4 b is not available on the loop. However, the unavailabletarget adaptor 4 b may still affect I/O operations of the source adaptor4 a, such as by preventing the loop 6 from completing a configuration.The target adaptor 4 b may be only temporarily unavailable. If (at block118) the get ID operation succeeded, i.e., the ID of the target adaptor4 b is included in the loop registry, then the device driver 22 a issues(at block 120) a reset request to the received ID of the target adaptor4 b to cause a reset of the target adaptor 4 b. Otherwise, if (at block118) the get ID request did not succeed, then the device driver 22 areturns to block 116 to issue additional get ID requests until the ID isreceived or the window timeout timer expires at block 104. The devicedriver 22 a may be unable to obtain the ID if the system 2 b includingthe target adaptor 4 b, or the I/O bays 52 a and 52 b in the case ofFIG. 2, is in the process of being reset.

In alternative implementations, the source adaptor 4 a may query thetarget adaptor 4 b over the loop 6 for the adaptor 4 b ID. With thelogic of FIG. 3, if the target adaptor 4 b becomes available after oneor more get ID requests fail at block 118 and before the windows timeouttimer expires at block 104, then the device driver 22 a would send thereset command to the adaptor 4 b to cause the adaptor 4 b to reset. If,during the reset operation, the target adaptor 4 b is unable toestablish communication with the system 2 b, then the target adaptor 4 bwould not come back online and would not be available on the loop 6until after the system 2 b resets or power cycles and corrects thecommunication problem with the target adaptor 4 b. If the target adaptor4 b becomes unavailable as a result of the reset, then the sourceadaptor 4 a and any other adaptors on the loop 6 can continue operatingon the loop 6 without concern that the problems at the off-line targetadaptor 4 b will cause problems.

In the described implementations, the device drivers 22 a and 22 bcommunicate an adaptor reset command. In alternative implementations, asystem reset or power cycle command may be communicated. Inimplementations such as FIG. 2 where the two adaptors are housed in thesame system, then a problem may occur at the I/O bay 52 a and 52 b levelincluding the adaptor, such that the adaptor, e.g., adaptor 54 b, cannotcommunicate with the system even though the adaptor 54 b can communicateon the loop. In such case, the device driver 72 a detecting the problemin the I/O bay in the same system 50 may perform the reset with respectto the specific I/O bay 52 b where the error is detected to avoidresetting or rebooting the entire system 50.

With the logic of FIG. 3, the device drivers 22 a, 22 b upon detectingerrors at another peer adaptor will begin monitoring I/O requests todetermine if any unacceptable I/O delays have occurred, i.e., I/O delaysthat extend the duration of the I/O delay timer. If no such unacceptableI/O delays occur, then the device drivers 22 a, 22 b would continueperforming I/Os and the system 2 a, 2 b housing the target adaptor wouldeventually reset without affecting I/O processing at the other adaptor 4a, 4 b. However, if an I/O request is delayed for the deemedunacceptable period, i.e., the I/O delay period, then a reset requestwill be sent to the adaptor 4 b including the target adaptor 4 b underthe assumption that the problems in the system 2 b including the targetadaptor 4 b are affecting the target adaptor's 4 b ability tocommunicate on the loop 6. In cases where the device driver 22 a, 22 bcannot communicate with the target adaptor, i.e., cannot obtain the IDof the target at blocks 116-120, then the device driver 22 a willcontinue to attempt to obtain the target adaptor 4 b ID by resending theget ID request until either the identifier is received so that a resetrequest can be transmitted at block 120 or the window timeout timerexpires at block 104 indicating that the system 4 b housing the detectedadaptor 4 b with the communication problem has had sufficient time toinitiate and complete an error recovery procedure.

The logic of FIG. 3 thus provides a greater degree of intelligentmonitoring in cases where there is a problem in the system including oneadaptor that will eventually result in a reset after a period of timethat is longer than an acceptable I/O delay time given the availabilityrequirements of the system. With the described implementations,unacceptable I/O delays are not experienced while waiting for the systemor I/O bay including the other adaptor to reset. The describedimplementations are particularly suited for situations where thedetected problem between the target adaptor 4 b and system 2 b includingthe adaptor is not likely to prevent I/O requests at other adaptors fromcompleting. The logic of FIG. 3 would allow one adaptor to continueprocessing I/O requests and wait for the detected adaptor to internallyreset without having to interrupt I/O processing.

FIGS. 4 a and 4 b illustrate logic implemented in the device drivers 22a and 22 b to account for the fact that the adaptor 4 a, 4 b in thesystem where the communication error is detected includes the masteradaptor for the loop 6. The logic of FIGS. 4 a and 4 b provides for amodification of the logic of FIG. 3 to reduce any delays associated withswitching the master if the master adaptor is in the target adaptor thatwill be subject to reset. The steps at block 200, 202, 204, 205, 206,208, 210, 212, and 214 are identical to the steps performed at blocks100, 102, 104, 105, 106, 108, 110, 112, and 114 in FIG. 3. The logicdiverges in FIGS. 4 a and 4 b to provide for switching the master ifnecessary to avoid any delays in altering the master configuration. Uponreceiving notification that an I/O timer expired (at block 214), thedevice driver 22 a performs the steps at blocks 216, 218, and 220 (whichare identical to steps 116, 118, and 120 in FIG. 3) to issue the commandto do the reset of the system/I/O bay where the error was detected. Asdiscussed, the reset may comprise a reset of the adaptor 4 b only, or inadditional implementations a power cycle command for the system 2 orpower boundary including the target adaptor 4 b.

Upon detecting the error at block 200, and concurrently with startingthe window timeout timer (at block 202) and monitoring for received I/Orequests (at block 206), an additional process is initiated at block 230(FIGS. 4 a and 4 b). With respect to FIG. 4 b, if (at block 230) theadaptor 4 b (housed in the system 2 b where the error was detected) isthe master, then a timer is started (at block 232) for a master switchtimeout period. The master switch timeout period is a timeout periodselected to change the master in a manner that avoids any masterswitching delays should the target adaptor 4 b in the system where theerror was detected be reset. In certain implementations, the masterswitch timeout period is set to a value less than the window timeouttimer and the I/O delay timeout to ensure that the master is switchedbefore the reset occurs following the I/O timer or window timerexpiring, so that the resetting process will not be further delayed byhaving to wait for a switch of the master to another adaptor when thetarget adaptor 4 b goes offline during the reset period. In this way,any I/O processing delays due to a master switch operation are avoided.

In implementations where the I/O delay timeout is greater than themaster switch timeout period, by the time the I/O delay timer expires,the master switch timer would have expired and triggered a change ofmaster operation at block 230, 232, 234, and 236 if the target adaptor 4b subject to the reset is the master. Otherwise, if the I/O delay timeris not greater than the master switch timer, then the reset processinitiated in response to the I/O delay timer expiring at block 214 wouldfurther reset the master as part of the reset process.

Upon receiving (at block 234) an alert that the master switch timerexpired, the device driver 22 a issues (at block 236) a command tochange the master to an adaptor having a next highest unique identifier(ID) port, e.g., adaptor 4 a, other than the adaptor 4 b subject to thereset. After block 236, the process started at block 230 to switch themaster ends. However, the process to monitor for any I/O requests andthe process waiting for the window timeout timer would remain pending toperform any further steps according to the logic described in FIG. 4 a.

The logic of FIGS. 4 a and 4 b provides additional checking over thelogic of FIG. 3 to avoid any delays incurred when changing themastership of the loop 6 from the target adaptor 4 b being reset toanother adaptor on the loop 6. In such case, the describedimplementations provide a master switch timeout period to ensure thatthe mastership of the loop 6 is switched before the target adaptor 4 bis subject to a reset command to avoid any I/O delays related toswitching the mastership on the loop 6.

Additional Implementation Details

The preferred embodiments may be implemented as a method, apparatus orarticle of manufacture using standard programming and/or engineeringtechniques to produce software, firmware, hardware, or any combinationthereof. The term “article of manufacture” as used herein refers to codeor logic implemented in hardware logic (e.g., an integrated circuitchip, Field Programmable Gate Array (FPGA), Application SpecificIntegrated Circuit (ASIC), etc.) or a computer readable medium (e.g.,magnetic storage medium (e.g., hard disk drives, floppy disks, tape,etc.), optical storage (CD-ROMs, optical disks, etc.), volatile andnon-volatile memory devices (e.g., EEPROMs, ROMs, PROMs, RAMs, DRAMs,SRAMs, firmware, programmable logic, etc.). Code in the computerreadable medium is accessed and executed by a processor. The code inwhich preferred embodiments are implemented may further be accessiblethrough a transmission media or from a file server over a network. Insuch cases, the article of manufacture in which the code is implementedmay comprise a transmission media, such as a network transmission line,wireless transmission media, signals propagating through space, radiowaves, infrared signals, etc. Of course, those skilled in the art willrecognize that many modifications may be made to this configurationwithout departing from the scope of the present invention, and that thearticle of manufacture may comprise any information bearing medium knownin the art.

In the described implementations, the monitoring logic was implementedin the adaptor device driver, which is an operating system program.Alternatively, the monitoring logic of FIG. 3 may be implemented withinthe adaptor hardware.

In the described implementations, the detected error condition whichtriggered the logic of FIG. 3 indicated a problem in the system or I/Obay including the adaptor, where the adaptor itself is functioningproperly. However, the logic of FIG. 3 may be initiated in response toother errors, such as internal adaptor hardware errors, line faulterrors, ACK time out error in not receiving the required acknowledgmentfrom the target adaptor, loss of synchronization error, code violationerror in decoding a character that is not data or is a specialcharacter, protocol error, sequence error, frame reject error, etc.Further details of the errors that may trigger the error recovery logicof FIG. 3 are described in the publication “Serial StorageArchitecture—Transport Layer 2 (SSA-TL2)”, Revision 5b, Working DraftT10.1/1147D (ANSI, Apr. 4, 1997), which publication is incorporatedherein by reference in its entirety.

In the described implementations, the reset sent by the device drivers22 a and 22 b is a reset to the adaptor and not other components in thesystem housing the adaptor. In alternative implementations, the resetmay cause a reset or power cycle of the entire system housing theadaptor. In implementations where the system including the adaptor wherethe error is detected includes multiple power boundaries, i.e., multipleI/O bays, then the reset may cause a power cycle to the specific powerboundary, i.e., I/O bay, including the target adaptor 4 b where theproblem is detected. Such a power boundary specific reset would notaffect other system components, such as other adaptors, on other powerboundaries, such as other I/O bays.

In the described implementations, the storage devices and adaptors areall connected on a loop topology, using a loop protocol such as SSA, theFibre Channel arbitrated loop protocol, etc. In alternativeimplementations, the two adaptors may be connected to the storagedevices in a non-loop topology. In the topology shown in FIG. 1, eachadaptor may directly communicate with any of the storage devices 8 a, 8b . . . 8 j. In additional implementations, one adaptor may have tocommunicate through another adaptor to reach a target storage device.

In certain implementations, the disk drives 8 a, 8 b . . . 8 j comprisedmagnetic hard disk drives. In alternative implementations, the storagedevices 8 a, 8 b . . . 8 j may comprise any storage device known in theart, such as optical disks, tapes, etc.

The preferred logic of FIGS. 3, 4 a, and 4 b describes specificoperations occurring in a particular order. In alternativeimplementations, certain of the logic operations may be performed in adifferent order, modified or removed. Moreover, steps may be added tothe above described logic and still conform to the describedimplementations. Further, operations described herein may occursequentially or certain operations may be processed in parallel.

The foregoing description of the preferred embodiments of the inventionhas been presented for the purposes of illustration and description. Itis not intended to be exhaustive or to limit the invention to theprecise form disclosed. Many modifications and variations are possiblein light of the above teaching. It is intended that the scope of theinvention be limited not by this detailed description, but rather by theclaims appended hereto. The above specification, examples and dataprovide a complete description of the manufacture and use of thecomposition of the invention. Since many embodiments of the inventioncan be made without departing from the spirit and scope of theinvention, the invention resides in the claims hereinafter appended.

1. A system for processing Input/Output (I/O) requests to a storagenetwork including at least one storage device and a system including afirst adaptor capable of communicating I/O requests to at least onestorage device, wherein the system including the first adaptor initiatesan error recovery procedure after a system timeout period has expired,comprising: a second adaptor capable of communicating on the storagenetwork; means for detecting an error in the system including the firstadaptor, wherein the first adaptor is capable of communicating on thestorage network after the error is detected; means for determiningwhether the first adaptor is designated a master of the storage networkafter the error is detected; means for starting a master switch timer,after detecting the error, that is less than the system timeout periodif the first adaptor is the master; and means for initiating anoperation to designate the second adaptor in the storage network as themaster if the first adaptor is the master in response to detecting anexpiration of the master switch timer, wherein a reset request is sentto an identifier of the first adapter after the master switch timerexpires.
 2. The system of claim 1, wherein the first adapter is in afirst computer system and wherein the second adapter is in a secondcomputer system.
 3. A computer readable medium including code forprocessing Input/Output (I/O) requests to a storage network including atleast one storage device and at least two adaptors, wherein each adaptoris capable of communicating I/O requests to at least one storage device,wherein the code when executed by a processor causes operations, theoperations comprising: detecting an error in a system including a firstadaptor, wherein the first adaptor is capable of communicating on thestorage network after the error is detected; determining whether thefirst adaptor is designated a master of the storage network after theerror is detected; starting a master switch timer that is less than asystem timeout period if the first adaptor is the master after detectingthe error, wherein an error recovery procedure in the system includingthe first adaptor is initiated after the system timeout period hasexpired; and initiating an operation to designate a second adaptor inthe storage network as the master if the first adaptor is the master inresponse to detecting an expiration of the master switch timer, whereina reset request is sent to an identifier of the first adapter after themaster switch timer expires.
 4. The computer readable medium of claim 3,wherein the first adapter is in a first computer system and wherein thesecond adapter is in a second computer system.